Surveying apparatus having a range camera

ABSTRACT

Embodiments described herein include a surveying apparatus for surveying a measurement scenery. The surveying apparatus may include a base defining a vertical axis; a support tiltable around the vertical axis; a telescope unit tiltable around the vertical axis and around a horizontal axis that is orthogonal to the vertical axis and comprises means for distance measurement; motor means for rotational driving of the support and the telescope unit; and angle determination means for detecting an orientation of the telescope unit with respect to the base. In some embodiments, the telescope unit comprises a first camera capable to take a visible image of the measurement scenery and/or means for capturing coordinates of 3D-points of the measurement scenery. In some embodiments, the surveying apparatus comprises a display capable to display at least a portion of the visible image taken by the first camera and/or at least a portion of the 3D-points.

The present invention relates to a surveying apparatus having a rangecamera and to a surveying method.

In prior art plural kinds of surveying apparatuses such as totalstations are known. Various means and methods are used for controllingsuch total stations. For instance, touch screen displays are known inorder to determine measuring points after an image of the measurementscenery has been taken. In order to determine a measurement point, theuser touches the image showing the respective point. By other means suchas edge extraction the measurement point chosen e.g. by the user tippinghis finger on the point, can be more accurately determined and marked onthe image. Then the measurement or surveying of the chosen point forinstance in a single surveying action or in the course of a surveyingpath can take place. That is, in this manner it is also possible tochoose a plurality of measurement points, for instance to create ameasurement path.

In some cases a detailed determination of a measurement point appearsdifficult because the common displays generally used with prior artsurveying apparatuses are rather small. Therefore, also the size of themeasurement scenery images is rather small. If merely a portion of themeasurement scenery image is shown, it is quite cumbersome to move theimage portion shown on the display to correspond to a desired area ofthe measurement scenery.

Thus, there is need to provide a surveying apparatus and a surveyingmethod capable to facilitate control steps performed by a user.

According to the invention, a surveying apparatus for surveying ameasurement scenery comprises

-   -   capturing means for capturing coordinates of 3D-points of the        measurement scene, and/or    -   a first camera that is capable to take a visible image of the        measurement scenery.

In particular, the surveying apparatus can be a 3D laser scanner that iscapable to capture coordinates of a multitude of 3D-points of themeasurement scenery, particularly a point cloud, or a total station thatis capable to capture coordinates of surveying points of the measurementscenery. Obviously, also a combination of a total station and a scannercan be used.

Furthermore, a display is provided which is capable to display at leasta portion of the visible image and/or the 3D-points. If a camera isprovided, the display preferably is capable to display three-dimensionalpoints captured by the capturing means in the visible image taken by thecamera. Preferably, the display is also capable to display pre-storeddigital terrestrial models of the measurement scenery and to overlay3D-points captured by the capturing means on this model.

Furthermore, according to the invention a range camera is directedtowards the display and capable to take range images of the userpositioned at the display. Furthermore, a controller is provided whichis capable to analyze the range images with regard to changes caused bya movement of the user, and to control the surveying apparatus on thebasis of the changes in the range image to perform predetermined tasks.

Range imaging in general is known as a technology which is used toproduce a 2D-image showing the distance to points in a scene from aspecific point. The resulting image, which is generally called rangeimage, has pixel values, which correspond to the distance of therespective target point at the object.

For instance, brighter values mean shorter distances or vice versa. Itis even possible to properly calibrate the sensor producing such a rangeimage which enables that pixel values can be given directly in physicalunits such as meters. For each of the pixels of the range image (rangepixels) one separate sensor capable to measure a distance is assigned.Since the distance of the target point assigned to the respective sensor(pixel) is known, the 3D-position of the target point can be exactlydetermined.

Thus, by using the range imaging technology, it is possible to identifyeach of measurement points of a user operating the surveying apparatus,and to even determine each measurement point's 3D-data. Here, ameasurement point of the user can be the end of an extremity such as ahand or a finger tip.

The term “range images” in the terminology of the invention generallymeans a sequence of range images shot, but can also mean a range imagestream taken by a streaming range camera.

According to the invention, a user operating the surveying apparatus bymoving his body or extremities, such as arms, legs, hands or feet, leadsto that in the range image of the user changes can be determined. Sincethe change of the 3D-position of an extremity of the user is recognizedby the controller, a corresponding movement of the user can bedetermined and an action or task corresponding to this certain movementcan be performed.

According to the invention, the display can be a 3D-display, which iscapable to display a 3D-image of the portion of the visible image.

The 3D-image can be generated by any known 3D-imaging method. Inparticular it can be one of the following methods: stereoscopy,auto-stereoscopy, computer generated holography or volumetric displaytechnique.

In particular, the volumetric display technique is preferred, becausehere a direct interaction between the user and the 3D-image can bepossible. That is, by volumetric display technique through 3D-images canbe shown which can be looked at from any arbitrary side. Furthermore, itcan be possible to position an object or an extremity directly in this3D-image. Thus, although the 3D-image is not a real object, it ispossible to touch the 3D-image.

Furthermore, although it is not visible for a watching person,stereoscopy can be used under certain circumstances. Since thistechnique requires special spectacles for the user to be able to see twosimilar images as one 3D-image, touching into the 3D-image in general ispossible. However, in order to determine the point of the 3D-image beingtouched by the user, the user's eye's exact position must be known,because the coordinates of the touched point in the 3D-space depend onthe user's position.

Since the 3D-image is produced by the controller, and all 3D-coordinatedata points of the displayed objects are known, it is possible tocompare these known 3D-coordinates with the 3D-coordinates of the movinguser known from the range image in order to determine whether the usertouches a certain object or target point in the 3D-display.

Advantageously, a map can be provided, in which predetermined changes ofthe range images corresponding to respective user movements are assignedto predetermined tasks, and the controller is capable to ignoreunassigned changes of the range image.

Since merely certain movements are assigned to predetermined tasks, thecomputing power can be reduced, because the controller is able toimmediately exclude movements that are not intended to be a command tothe surveying apparatus.

Furthermore, advantageously, learning means can be provided to learnrange image changes corresponding to respective user movements in orderto be assigned to respective predetermined tasks and/or to be stored ina map. That is, according to the invention, for the user it is possibleto define certain movements and to assign these movements to apredetermined task. Furthermore, it is possible to store these movementsand the corresponding predetermined tasks in a map. Thus, theoperability of the surveying apparatus can be improved.

According to the invention, a surveying method using a surveyingapparatus comprises

-   -   capturing coordinates of 3D-points of a measurement scenery with        the surveying apparatus and/or taking an image of the        measurement scenery with a first camera, and    -   displaying at least a portion of the image and/or the 3D-points        of the measurement scenery on a display.

Furthermore, the method comprises

-   -   taking a sequence of range images of a user positioned at the        display,    -   analyzing movements of the user on the basis of changes in the        individual range images of the sequence, and    -   controlling the total station to perform tasks corresponding to        the user movements.

Thus, according to the invention, since the movements of the user can beanalyzed on the basis of changes in the individual range images taken bythe range camera, it is possible to assign certain movements tocorresponding tasks of the surveying apparatus in order to control thesurveying apparatus. Thus, it is not necessary to depend on a rathersmall touch screen display, but a quite large display such as a big TVscreen or a display composed of plural TV screens can be used to displaythe portion of the image of the measurement scenery, while the user'smovements for controlling the surveying apparatus in relation to thedisplay can be analyzed. Preferably, also digital terrestrial models,for instance pre-stored images and/or point clouds of the actualmeasurement scenery, can be displayed on the display and combined withimages or 3D-points of the measurement scenery captured by the firstcamera or the capturing means, respectively.

Advantageously, the portion of the image and/or the 3D-points of themeasurement scenery being displayed is a 3D-image.

Advantageously, plural surveying apparatuses can be provided atdifferent locations to take images of the same measurement scenery fromdifferent viewpoints. In this case, the portion of the image and/or the3D-points of the measurement scenery is composed from image informationfrom all of the images of the different surveying apparatuses.

The features of the above paragraph enable improving the quality of the3D-image, because the measurement scenery is taken from differentlocations. Accordingly, areas not directly visible from the onesurveying apparatus can be shot or surveyed by one of the othersurveying apparatuses. Therefore, the quality of the 3D-image can beimproved. In order to exchange data between the different surveyingapparatuses, preferably wireless communication methods such as radio oruse of mobile networks e.g. short message service or others can beemployed.

The 3D-image can be generated by one of stereoscopy, auto stereoscopy,computer generated holography or volumetric display technique.

In particular the volumetric display technique is preferred fordisplaying the 3D-image, because this technique enables an interactionbetween a user and the image in a manner that the user can kind of touchthe objects displayed in the 3D-image.

Advantageously, the method of the invention can comprise a learning stepin which changes of the range images caused by certain user movementsare assigned to predetermined tasks.

This enables the user to define certain preferred movements in order toteach the surveying apparatus tasks corresponding to the certainpreferred movements. It is even possible to store these movements andthe assigned tasks in a map for later use.

Advantageously, the portion of the image and/or the 3D-points of themeasuring scenery can be analyzed and possible target points can bemarked by overlaying the portion of the image of the measuring scenerywith marking symbols.

Thus, when operating the surveying apparatus, due to the analyses of themeasurement scenery and the markings of possible target points, the usercan much easier choose a certain target point in the 3D-image. It can beeven possible to perform a command to move a marking in case thecorresponding target point is not properly assigned.

Furthermore, advantageously, a target point chosen by the user can haveits marking highlighted, and a predetermined user movement can causethat another target point is also highlighted.

Another target point can be any of the target points displayed in the3D-image, that is, an adjacent target point of the firstly highlightedtarget point or a target point being more distant. Furthermore, byhighlighting plural target points one after the other, a measurementpath for performing subsequent measurements can be defined. It is alsopossible to extinguish the highlighting of some or all of the markingsin order to remove target points from the measurement path.

Advantageously, a predetermined user movement can cause an increase or adecrease of the size of the portion of the image and/or the 3D-points ofthe measurement scenery. Furthermore or alternatively, another usermovement can cause a rotation of the portion of the image and/or the3D-points of the measurement scenery.

Thereby it is possible to zoom into the 3D-image or to zoom out of it.Furthermore, in order to facilitate access to certain objects, it can bepossible to rotate the 3D-image. In particular this rotation isfacilitated, if plural surveying apparatuses are provided. If there isonly one surveying apparatus provided, areas being not visible for theone surveying apparatus cannot be properly displayed.

The invention in the following will be described in detail by referringto exemplary embodiments that are accompanied by figures, in which:

FIGS. 1 a and 1 b are schematic views of a total station and a laserscanner as two exemplary embodiments of a surveying apparatus accordingto the invention;

FIG. 2 is a schematic view of a measurement scenery with a surveyingapparatus according to the invention;

FIGS. 3 a and 3 b are schematic images of RIM images of the userperforming certain movements;

FIG. 4 is a schematic view of a user operating a total station using a2D-image; and

FIG. 5 is a schematic view of a user operating a total station using a3D-image.

Preferred embodiments of the invention will be described on the basis ofFIGS. 1 to 5.

FIG. 1 a shows a total station as a first embodiment of a surveyinginstrument according to the invention. The depicted exemplary totalstation is adapted for measuring horizontal and vertical angles anddistances to a remote target object.

The total station is provided on a tripod 36, a base 31 of the totalstation being directly fixed on the tripod 36. The main part 30 of thetotal station is rotatable relative to the base 31. The main part 30comprises a support 32, in this exemplary embodiment being formed by twocolumns. Between the columns a telescope unit 33 is supported tiltablyaround the horizontal axis. Furthermore, the main part 30 comprisesdisplay and controlling means 35 which can be suitable in a known mannerfor controlling the total station and for processing, displaying andstoring measurement data.

The telescope unit 33 is arranged on the support 32 tiltably around ahorizontal axis and thus can be rotated horizontally and vertically withrespect to the base 31. Motor means (not shown) are provided forperforming the required tilting movements for the alignment of thetelescope unit 33.

The telescope unit 33 can be built as a component unit, wherein anoptical system, a coaxial camera sensor, an eyepiece 34 and a graphicsprocessor are integrated in a common telescope unit housing. Thetelescope unit 33 can be aimed at a target object so that the distancefrom the total station to the target object can be detected by means ofelectronic sensors. Furthermore, electronic sensor means (not shown) areprovided for detecting an angular orientation of the main part 30relative to the base 31 and of the telescope unit 33 relative to thesupport 32. The data are sent to the display and controlling means 35and processed so that the position of the target point relative to thetotal station is detectable, displayable and storable by the display andcontrolling means 35.

FIG. 1 b shows a sectional view of a laser scanner as a secondembodiment of a surveying instrument according to the invention. Thedepicted exemplary laser scanner comprises a stator 41, a rotor 42 thatis mounted on the stator 42 to be rotatable about a first rotationalaxis, and a rotary body 43 that is mounted on the rotor 42 to berotatable about a second rotational axis. In the rotor 42 a laser source44 and a detector 45 are arranged. On two sides of the rotary body 43between the rotor 42 and the rotary body 43 optical links 46 a,b areprovided on the second rotational axis in such a way that emission lightcan be introduced by the laser source 44 into the rotary body 43 via thefirst optical link 46 a and reception light can be discharged from therotary body 43 via the second optical link 46 b. A first rotary drive 47a drives the rotor 42 and a second rotary drive 47 b drives the rotarybody 43. Two goniometers 48 and evaluation electronics 49 which areconnected to the laser source 44 and the detector 45 allow associationof a detected distance with a corresponding direction. Obviously, alsoother forms of 3D laser scanners, for example having a mirror instead ofa rotary body with optical links, are equally adequate.

This far the surveying instruments shown in FIGS. 1 a and 1 b are knownfrom prior art. According to the invention, additionally a display and arange camera are provided (not shown). The range camera is directedtowards the display and capable to take a range image of a userpositioned at the display.

FIG. 2 is a schematic view of a measurement scenery. Main objects in themeasurement scenery are a church 7, a house 9, a tree 11 and a mountainpeak 13. A total station 1 is provided as an example for a surveyingapparatus, comprising a first camera for taking a real image of themeasurement scenery. Furthermore, the total station 1 is provided with arange camera, in particular a RIM-camera.

The range camera is directed to a display 3 located near the totalstation 1 and takes a sequence of range images. Movements of a user 5located at the display 3 can be determined as changes in the rangeimages taken by the range camera.

For this purpose, a control unit provided in the total station 1 candetermine certain movements of the user 5 and have the total station 1to perform certain surveying tasks and other tasks. Some of these taskswill be described below.

Thus, by moving his extremities or performing other movements, the user5 can give commands in order to control the total station 1. In theimage, the measured distances are displayed on the basis of sixbrightness levels. Brightness level 1 means that the point displayedwith the brightness level is closest, while brightness level 6 meansthat the corresponding displayed point is most distanced.

FIG. 3 a shows an initial position of the user 5 standing in about themiddle of the range which can be displayed. Thus, the images displayingthe user 5 have brightness level 4.

FIG. 3 b shows a change in that the user 5 has lowered his left armtowards a horizontal position and has moved it backwards. Furthermore,the user 5 has moved his right arm fully downwards and forwards.Accordingly, the left hand of the user 5 is most distanced from theRIM-camera in the total station 1 and the pixels of the range imagecorresponding to the left hand are quite dark (brightness level 6). Incontrast, the right hand of the user 5 is quite near to the RIM-camerain the total station 1 and, thus, the pixels corresponding to its lefthand are quite bright (brightness level 1).

Thus, by determining the user's movements on the basis of the changes ofthe range images, the controller issues certain commands in order tohave the total station 1 perform various actions and/or surveying tasks.For instance, the user can define a measurement path, can defineadditional measurement points, can zoom into the image or zoom out ofthe image, can switch on images taken by additional total stations orcan quit the measurement.

In order to perform an accurate control by the user 5 and to preventerroneously given commands, in the range image a certain sectioncorresponding to a control zone 15 marked by a dashed line is defined.For determining the user's 5 movements, the user 5 must be present inthis control zone 15. Accordingly, if the user 5 steps out of thecontrol zone 15, although changes of the user's 5 posture can still bevisible in the range images, the controller will ignore the changes and,thus, the total station 1 will not react upon the user's 5 commands.

Preferably, the control zone 15 can be in the form of a small platformor depression.

In FIGS. 3 a and 3 b, the white fields correspond to points falling outof the measuring range of the RIM-camera.

FIG. 4 shows a display 3 displaying the measurement scenery of FIG. 1 ina two-dimensional manner. The user 5 is located close to the display 3,in order to be able to point at the objects displayed. For instance, theuser may point at the top of the church 7, at the tree 11, at the house9 or at the mountain peak 13.

The presentation in a two-dimensional display, as is the case in thisembodiment, even enables to show moving elements such as the house 17 orthe traction engine 19. Therefore, e.g. a person 21 showing a surveyingpole at a predetermined location and then moving to another location canbe also displayed. In order to enable the displaying of moving elements,the first camera in the surveying apparatus 1 has to take an imagestream which is then displayed on the display 3.

Since the user 5 is located at the display 3, he is in a position topoint to a chosen target point with one hand. This movement isrecognized due to changes in the range image of the user. Since the3D-position of the user's hand is exactly known, it can be assigned to acorresponding pixel in the display 3. Therefore, it is possible toexactly determine the object the user 5 is pointing at. By moving hisother arm, the user 5 can initiate an action such as marking the chosentarget point by a marking 23,25,27 in the image. Furthermore, certainsimilar movements serve for marking other target points by additionalmarkings. Then, the user can initiate a surveying action along thesurveying path e.g. along the markings 23,25,27.

Other possible actions are zooming into the picture or zooming out ofthe picture removing markings, shifting markings etc. If plural totalstations are provided at different locations, by a certain movement theuser can initiate switching from one total station's view to anothertotal station's view. It is also possible to split the image in thedisplay 3 to two or more images in order to display the images ofdifferent total stations at the same time.

Instead of using actual images of the first camera it is also possibleto have a pre-stored digital terrestrial model of the measurementscenery displayed on the display 3, and to overlay measurement points onthe model.

FIG. 5 shows an embodiment, in which the display is a 3D-display. Thereare certain 3D-displaying techniques available, such as stereoscopy,auto-stereoscopy, computer-generated holography or volumetric displaytechniques. In this embodiment, preferably a 3D-technique using voxelsis used. In 3D-displaying technique, a voxel (volume pixel) is athree-dimensional object point, corresponding to a pixel in a 2D-image.

In spite of the technical effort to provide the 3D-image in the form ofvoxels, this technology has the advantage that the user can directlytouch the objects in the 3D-image. With other kinds of 3D-techniques,this would not be possible.

The display 3 in FIG. 5 displays the measurement scenery of FIG. 2. Inorder to obtain the data for the 3D-display, known techniques such asstereoscopy, auto stereoscopy, computer-generated holography orvolumetric display technique or range imaging can be employed. In caseof range imaging, a second range camera is to be used for obtaining the3D-data of the measurement scenery.

As was described with regard to FIG. 4, by touching an object in the3D-display, the user 5 can mark certain target points, can initiate asurveying action e.g. via a chosen surveying path or even zoom into orout of the 3D-image. Furthermore, the user can cause a rotation of the3D-image as well. In order to display areas not visible for the totalstation, advantageously, additional total stations can be provided atdifferent locations.

While the description of FIG. 4 pertains to a 2D-display, it is to benoted that even here displaying in a 3D-manner is possible if thestereography technique is provided. That is, by providing two images ofthe same scenery, in which the image axes are slightly different, athree-dimensional effect can be made available, for instance if the userwatches the display from a certain angle or wears special spectacles.

In this case it is possible, to give the user a 3D-display, althoughanother person watching the display from the outside would merely seethe two images shifted against each other. The user, however, has aclear 3D-image and can even interact with the 3D-image as is the casewith the 3D-image based on the volumetric display technique.

While the invention has been described with reference to presentlypreferred embodiments, it is to be noted that the scope of the inventionis defined by the attached claims.

1-15. (canceled)
 16. A surveying apparatus for surveying a measurementscenery, the surveying apparatus comprising: a base defining a verticalaxis; a support tiltable around the vertical axis; a telescope unittiltable around the vertical axis and around a horizontal axis that isorthogonal to the vertical axis and comprises means for distancemeasurement; motor means for rotational driving of the support and thetelescope unit; and angle determination means for detecting anorientation of the telescope unit with respect to the base, wherein thetelescope unit comprises a first camera capable to take a visible imageof the measurement scenery and/or means for capturing coordinates of3D-points of the measurement scenery, wherein the surveying apparatuscomprises a display capable to display at least a portion of the visibleimage taken by the first camera and/or at least a portion of the3D-points, respectively, and wherein a range directed towards thedisplay and capable to take a range image of a user positioned at thedisplay, wherein a controller is provided, which is capable to analyzethe range image with regard to changes caused by a movement of the user,and to control the surveying apparatus on the basis of the changes inthe range image to perform predetermined tasks.
 17. The surveyingapparatus according to claim 16, wherein the range camera comprises aRIM-camera.
 18. The surveying apparatus according to claim 16, whereinthe display is a 3D-display capable to display a 3D-image of the portionof the visible image and/or the 3D-points.
 19. The surveying apparatusaccording to claim 18, wherein the 3D-image is generated by stereoscopy,auto stereoscopy, computer generated holography or volumetric displaytechnique.
 20. The surveying apparatus according to claim 18, whereinthe display is capable to display a digital terrestrial model of themeasurement scenery.
 21. The surveying apparatus according to claim 18,wherein a map is provided in which predetermined changes of the rangeimage corresponding to respective user movements are assigned topredetermined tasks, and the controller is capable to ignore notassigned changes of the range image.
 22. The surveying apparatusaccording to claim 18, wherein learning means is provided to learn rangeimage changes corresponding to respective user movements in order to beassigned to respective predetermined tasks and/or to be stored in a map.23. The surveying apparatus according to claim 18, wherein the surveyingapparatus is a total station.
 24. A surveying method using a surveyingapparatus, the surveying apparatus comprising: a base defining avertical axis; a support tiltable around the vertical axis; a telescopeunit tiltable around the vertical axis and around a horizontal axis thatis orthogonal to the vertical axis and comprises means for distancemeasurement; motor means for rotational driving of the support and thetelescope unit; and angle determination means for detecting anorientation of the telescope unit with respect to the base, the methodcomprising: taking an image of a measurement scenery with a first cameraand/or capturing coordinates of 3D-points of a measurement scenery,displaying at least a portion of the image and/or the 3D-points,respectively, of the measurement scenery on a display, taking a sequenceof range images of a user positioned at the display, analyzing movementsof the user on the basis of changes in the individual range images ofthe range image sequence, and controlling the surveying apparatus toperform tasks corresponding to the user movements.
 25. The surveyingmethod according to claim 24, wherein the portion of the image and/orthe 3D-points, respectively, of the measurement scenery is a 3D-image.26. The surveying method according to claim 25, wherein the 3D image isdisplayed in a manner that a user can access image points.
 27. Thesurveying method according to claim 24, wherein plural surveyingapparatuses are provided at different locations to take images and/or3D-points of the same measurement scenery from different viewpoints, andwherein the portion of the image and/or the 3D-points, respectively, ofthe measurement scenery is composed from image information and/or3D-points information, respectively, from all of the images and/or3D-points, respectively, of the different surveying apparatuses.
 28. Thesurveying method according to claim 24, wherein the 3D-image isgenerated by one of auto-stereoscopy, computer generated holography orvolumetric display technique.
 29. The surveying method according toclaim 24, further comprising a learning step in which changes of therange images caused by certain user movements are assigned topredetermined tasks.
 30. The surveying method according to claim 29,wherein the portion of the image of the measurement scenery is analyzed,and possible target points are marked by overlaying the portion of theimage of the measurement scenery with marking symbols.
 31. The surveyingmethod according to claim 29, wherein a marking corresponding to achosen target point is highlighted and a predetermined user movementcauses to highlight a marking of an second target point.
 32. Thesurveying method according to claim 24, wherein a predetermined usermovement causes an increase or decrease of the size of the portion ofthe image of the measurement scenery and/or causes a rotation of theportion of the image of the measurement scenery.
 33. The surveyingmethod according to claim 24, wherein a digital terrestrial model of themeasurement scenery is displayed on the display.